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ABSTRACT 



This paper describes the processes and techniques used to 
improve and streamline the standard student reports used at Purdue University 
(Indiana) . Various models for analyzing reporting processes are described, 
especially the model used in the study, the Shewart or Deming Cycle, a method 
that aids in continuous analysis and improvement through a four step process 
-- plan, do, check, and act. The model was applied to enrollment reports 
(using Microsoft Excel for calculations) , a database for tracking ad hoc 
information requests was developed (using Microsoft Access database 
software) , the statewide enrollment report was improved by using Microsoft 
Access, and the quality of reports was improved by using survey information. 
The paper concludes that utilizing appropriate technology within the Deming 
model has reduced the amount of labor needed for creating reports and 
improved the accuracy and timeliness of the reports. (Contains 16 
references.) (DB) 
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Effective Report Preparation: Streamlining the Reporting Process 



Abstract 

With improved technology and the increased demand for more information in a timely 
manner, preparing reports quickly and accurately has become increasingly important for 
institutional researchers. Preparation of reports was improved to take advantage of the 
most efficient technology and to better meet user needs. Specifically addressed are ways 
in which reports were streamlined including use of Microsoft Access software for 
tracking ad hoc data requests and preparing an enrollment report from begirming to end; 
using SAS/CONNECT to allow for more direct access to mainframe data; and surveying 
users to provide insight on how to improve an academic performance report. 



Effective Report Preparation: Streamlining the Reporting Process 

Introduction 

Utilizing the most effective technology to meet reporting needs and determining 
the most efficient way to prepare reports is an important and on-going concern for 
institutional researchers (Krotseng, McLaughlin, & Eimers, 1998). Developing a strategy 
to evaluate reporting activities and using methods of report preparation that take fullest 
advantage of current technology help to meet this goal (McLaughlin, Howard, & 
McLaughlin, 1998; Sanford, 1995). 

One of the top priorities of the student research area at our institution in recent 
years has been to streamline reports. Reports on enrollment, degrees conferred, academic 
performance, and retention are prepared on an annual or semester basis. In the not too 
distance past, some reports were prepared by manually entering data into word 
processing documents from hard copy reports, and in some instances, manually doing the 
calculations. While every institution has unique needs and ways in which reports are 
prepared and utilized, there are common processes that must be understood to take 
fullest advantage of technology and to meet user needs. Receiving accurate and timely 
reports is critical for every institution of higher education. Institutional researchers 
particularly understand the importance of finding better ways to prepare and present data 
to meet the needs of decision-makers. 

In order to reduce the time allocated for report preparation, processes were 
created that utilized appropriate technology to meet the university goal of doing more 
with less. The process used to streamline our student report preparation has reduced the 
amount of labor needed for creating reports and improved the accuracy and timeliness of 
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the reports. Surveying our users has given us information that has been used to better 
serve their needs. This paper covers the processes and techniques used to improve and 
streamline the standard student reports used at our institution. 

Models of Analyzing Reporting Processes 

Advances in technology have greatly impacted the way we work. More data are 
available today than ever before and expectations are high for accurate reports prepared 
in record time. There are several concepts of analyzing reports or processing decisions to 
develop new ideas and overall streamlining. Below are some of the models we found 
useful. 

McLaughlin, Howard, Balkan & Blythe (1998) have identified key elements for 
those who work with information. The first key element is that change is perpetual. 
Second, as institutional research professionals, we must embrace change or we will 
become obsolete. Finally, ideas are used as a key source of influence and change. 
Throughout the implementation of these steps we have utilized two primary learning 
disciplines described by McLaughlin, et al. One of which was a shared vision. A shared 
vision is when team participants hold a deep commitment to a commonly held purpose, 
such as streamlining report preparation. We are bound together by this shared aspiration. 
The discipline of team learning occurs with the alignment of the individual team 
members in the process of working together toward the next higher level of awareness 
(McLaughlin, et al., 1998). 

A model, by R.A. Howard (1988), that looks at the analysis process states that 
quality of the decision is guaranteed by performing the actions required in each step. The 
very basis of a vision is the steps it follows. Although the team members' "shared vision" 
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may increase the quality, just having steps to follow helps the process. Although the 
steps may be considered a formality, "the mere discipline of getting the people together 
and working through the process will yield benefits and continuous improvement" 
(deKlerk, 1994, p. 21). 

Another model of the analysis process is by J.E. Matheson (1992). Matheson 
(1992) identifies six links needed to obtain a quality decision in his "decision quality 
chain." The decision quality chain outlines steps taken by the team members. Link 1, the 
Appropriate Frame, views the problem. The correct perspective, such as strategic 
problem or operational problem, must be identified, as the real problem, not the apparent 
one. Link 2, Creative, Doable Alternatives, creates a set of alternatives. This is more 
than simply another concept, but a well thought out formula. Link 3 is Meaningful, 
Reliable Information; useful information is required to make relevant decisions; The 
information will primarily influence the outcome of the decision and the uncertainty 
associated with the factors. Link 4, Clear Values and Trade-offs explain end results in 
clear, expressed terms of values and the relative importance of each value. Link 5 is 
Logically Correct Reasoning. Analysis and determination of the value of information is 
completed. In Link 6, Commitment to Action, the decision making process should be 
followed through with implementation (deKlerk, 1 994). 

Another element that we use in analyzing our reporting process is evaluation. A 
model developed by Stufflebeam (1971) adds several practical criteria necessary to the 
definition of evaluation. These criteria are relevance, importance, scope, credibility, 
timeliness, pervasiveness, and efficiency. In addition to the criteria, there are three major 
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types of evaluation. Frenchtling, Stevens, Lawrenz, & Sharp (1995) suggests three major 
types of evaluation prototypes: 

1 . Planning Evaluation: A planning evaluation assesses the understanding of 
project goals, objectives, strategies and timelines. 

2. Formative Evaluation: A formative evaluation assesses ongoing project 
activities. It consists of two types: Implementation Evaluation and Progress 
Evaluation: 

a) An Implementation Evaluation assesses whether the project is being 
conducted as planned. 

b) A Progress Evaluation assesses the progress made by participants in 
meeting the project goals. 

3. Summative Evaluation: A Summative Evaluation assesses project success - 
the extent to which the completed project has met its goals (Frenchtling, et ah, 
1995). 

We have used the Summative Evaluation technique to evaluate our reports. In this 
category evaluation is determined successful if it meets the project's original goals. 
Frenchtling, et al. also suggested five phases for conducting an evaluation: 

1 . Develop Evaluation Questions 

2. Match Questions with Appropriate Information-Gathering Techniques 

3. Collect Data 

4. Analyze Data 

5. Provide Information to Interested Audiences 

(Frenchtling, et ah, 1995) 
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The five phases were found useful when evaluating the results of our new streamlined 
report processes. 

However, the theory or model we found most useful and applicable is probably 
the most recognized model, which is the Shewart or Deming Cycle. The Shewart Cycle 
renamed the Deming Cycle, is a method that aids in continuous analysis and 
improvement. It is composed of four basic stages that occur over time: a Plan stage, a Do 
stage, a Check stage, and an Act stage (Sherkenback, 1 992). A decision-maker 
recognizes an opportunity and develops a plan (1®‘ stage - Plan), she tests the theory by 
implementing the plan on a small scale or a trial bases (2"** stage - Do), she monitors the 
effects and observes the test results (3^** stage - Check), and acts on the opportunity and 
takes appropriate actions (4*** stage - Act). Since these actions lead to a new or revised 
plan, the cycle is a never-ending improvement. A diagram of the Shewart/Deming Cycle; 
is shown below. 




In the first stage. Plan, a strategy is developed. A collection of data is helpful for 
what must be accomplished. The data can assist when determining the effects of 
manipulating or changing the streamlining process. When the second stage, the Do stage, 
is conducted, a plan should be implemented in a small scale. Once the team members 
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have been educated and trained, the plan may be set into motion. In the Check stage, the 
plan is monitored to answer two questions. First, is the process behaving according to the 
plan and second, are the effects of the plan creating problems or improvements? The 
results of the studies in this Check stage lead to the Act stage. The purpose of the Act 
stage is to implement modifications to the overall process. No matter what, the cycle 
returns to the Plan stage. If it was learned that the process did not have the desired 
outcome, the cycle returns to the Plan stage to search for alternatives. If it did produce 
the desired results, it will lead back to the Plan stage to determine the optimal levels of 
the process improvements. Consequently, the cycle continues in the never-ending 
improvement of the process (Gitlow, Gitlow, Oppenheim, Oppenheim, 1989). 

Our own method of analyzing the report process incorporates all of the elements 
listed above. Our attempt to change grew out of the idea that we wanted to streamline 
our reporting processes to take better advantage of technology. Ultimately, these ideas 
evolved through a number of steps: 

Identify Need to Change 

Define Current Processes i ’ 

Identify Problem Areas 

Brainstorm How to Correct Problems 

Investigate Alternatives 

Develop a Process for Implementation 

Evaluate the Effectiveness of Changes Made 

As one can see, several elements from the above mentioned models were 
incorporated into our own method. We found the Deming model straightforward. 
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complete, and applicable to our approach. (However, we skip the trial run and implement 
stage four. Act, before stage three. Check.) Our approach fits within the Deming model. 
Within first stage. Plan, we identified the necessity of upgrading, because as McLaughlin 
et al. (1998) noted, change is perpetual and necessary. During this stage we also defined 
the current process, similar to Matheson's (1 992) viewing the problem in the correct 
perspective. Matheson (1992) and Deming noted the need to collect data, which we did 
as a part of identifying the problems. As Howard (1988) stated, we found that just having 
outlined steps helped us organize our plan. As in Deming's second stage. Do, we 
brainstormed solutions to possible problems. Creating alternatives was considered 
critical in Matheson's model. McLaughlin, et al. also noted that aligning team members 
in a shared vision is necessary in meeting the goal. In the Act stage, we implemented our 
idea by following the steps we had created, as Matheson recommends, and in the Check 
stage, we evaluated the effectiveness of the process. The evaluation theories of 
Stufflebeam (1971) and Frenchtling et al. (1995) helped us organize our evaluation to . 
determine if we were meeting our original goals. . 

Implementation Examples 

As noted earlier, technology has greatly impacted Institutional Research. 

Although it has become easier to create reports in some ways, expectations have also 
increased. Also, technology is continually changing and improving. Technology that 
was used ten or 15 years ago to prepare reports is likely to be woefully inadequate today. 
There has been dramatic change. Determining the best technology for preparing a 
specific report can be a significant challenge. Sometimes the choice of which technology 
to use is clear, sometimes it is not. Much is dependent on the source of the data, the skill 
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and training levels of the staff available, the time available to understand and to use the 
technology, and, of course, the capabilities of technology itself With the advances in 
technology and the increased demands for information, it is imperative that tools to 
translate data into reports be the most effective and efficient possible. 

The Never-ending Improvement of an Enrollment Report 

In the not too distance past, enrollment reports prepared each semester at our 
institution required a significant amount of manual data entry. “Official” hard-copy 
paper reports were generated from Cobol programs on the mainframe computer. These 
extensive reports were then summarized into a report that was condensed in format, was 
more appealing to the eye, and was significantly shorter. The data from the “official” : 
reports were read from the hard-copy and manually entered into a word processing 
document with the calculations done by hand. ; 

A big technology advance was made several years ago by moving the report from 
the word processing document to a Microsoft Excel file. This allowed the calculations to 
be done by the software, but still required manual data entry. Over the years, 
improvements to the spreadsheet has been made by creating links so that information that 
is reported on more than one spreadsheet need not be entered multiple times. Checks 
have been built into the spreadsheets that check for data accuracy and consistency. 

Although this has been a big improvement, it has not been enough. Manually 
entering data into a spreadsheet with all the technology available today is not sufficient. 
Through the process described above, we have identified the problem areas, investigated 
the alternatives, and developed a process for implementation. We want to be able to 
update the spreadsheet directly from a computer file. This will require working with staff 
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from the computing center who maintain the “official” report programs so that the same 
data source can be made available to us. Once we have the source data, we will use SAS 
to extract the data from the mainframe to feed into a Microsoft Access database and then 
manipulate the data in Access to get it into the appropriate Excel format. We want to 
retain the format as it currently exists in Excel and have decided not to use Access or 
SAS to create the report at this time. However, these may be viable alternatives in the 
future. It truly is an on-going process. 

Developing a Database for Tracking Ad Hoc Information Requests 

One of the most valuable new reporting tools is Access, a Microsoft database 
product. This product has increased efficiency and decreased the amount of time for . r 
report preparation and manual labor. The potential for using Access for reporting 
purposes at all stages of report preparation is virtually limitless. 

With responsibility for responding to ad-hoc requests which from a wide variety 
of internal and external constituents, keeping track of requests is essential. In the past, 
our tracking was done by filling out a paper form for basic information including who, ; 
when, what, and how. Once the forms were collected at the end of the month, a tally was 
created to see what topics were the most frequently requested and how long the requests 
took to complete. 

Two years ago, an Access database was created to store the information on these 
ad-hoc requests. We recognized the need to move this "paper-pushing" method online. 
We defined the process and identified key sections of tracking the ad hoes. It was 
obvious that a database would best fit oitr process, and Access was available to us. After 
the implementation we evaluated the new system of tracking by simple comparison. It 
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went above and beyond our original goals. The information can now be accessed from 
the database and summary reports can readily be generated. However, the greatest 
benefit of moving the ad-hoc requests to Access has been the organization of the 
requests. Instead of sorting through a file of several requests, searches can be done in a 
few seconds. Another advantage of using a database is being able to search for a request 
in several ways. If an ad-hoc report was requested earlier by another person or another 
department, the request can be found easily, saving time sorting through the filing 
cabinet. 

Using an Access Database for Statewide Enrollment Reporting 

Access has also been used to recreate a statewide enrollment report. Enrollment 
data are received from most of the public and private institutions of higher education in 
the state and a summary report is prepared annually, which includes historical - ^ 

information. In the past, this report was created using multiple software applications, ■ 
using both mainframe and personal computers. Instead of using a combination of 
mainframe files, Microsoft Word, and Excel, the entire process now has been moved into 
Access. 

Before using Access, the data were entered manually into mainframe files and 
several programs were run to check the data and create the data tables. These tables were 
exported into Excel (which was an improvement from using a word processor where 
manual calculations were used previously) and then cleaned up into the format desired 
for the final report. The mail merge capability of Word was used to generate letters and 
create labels for mailings as well as for preparing the title page and the table of contents 
page. 
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Because of an upcoming deadline for removing the reporting language used in the 
first step of the manipulation of the data, we knew we had to convert at least part of the 
process. We analyzed the process and outlined the important components. There were 
several alternatives, but in the end we choose Access. Our decision was based on the 
idea to use only one tool for the entire report. Not only was this due to training, but we 
were also concerned that if we used multiple software packages, compatibility problems 
may arise. 

The project to move the report to Access was undertaken in several stages, the 
first of which was to create data tables. Once the tables were completed, queries were 
created to check and clean the data. A report was produced to recreate the tables that 
were once created using the older report technology. Also, the letters sent to recipients of 
the report are stored in Access and merged with the address data table. The ultimate step, 
which has not yet been completed, will be to use the web for data input. The institutions 
will be able to enter their own information, which can then be downloaded, directly into 
Access. Once that step is finished, the entire report will be in Access and it will be 
updated by the institutions. 

We have only done the initial report so our evaluation of it is somewhat limited. 
Although, the report section was more complicated than anticipated, overall, our goals 
were met and we found the whole process more efficient. The benefits of having the 
whole report in Access have been not only more efficiency, but also the ability to easily 
add data history and graphs. 
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Using Survey Information to Improve the Quality of Reports 



As part of the overall effort to improve reports, a survey was conducted of the 
recipients of a report on academic performance of undergraduate students which is 
prepared each fall and spring term. The benefit of understanding users needs is an 
essential component of any effective report (Frost, & Beach, 1994). 

Preparation of the Academic Performance Report takes a considerable amount of 
time and effort, yet we rarely received ad hoc requests that required information from this 
specific report. We tried to figure out what was the most important information to 
provide and if there was anything that was unnecessary included in the report. A written 
questionnaire was given to 57 recipients of the report concerning the portion they used 
most and least often, the frequency of use, and their opinions about a Web site option for 
posting the report. A response rate of 56% was obtained. The results of the smvey 
showed that many of the users were satisfied with the format and content of the report, ; ^ 
but their use was infrequent. Additionally, and significantly, respondents indicated that 
publishing this report on the world wide web would meet their needs. The survey was a 
way to see how we could improve the quality of the Academic Performance Report, and 
served as a check, as suggested by the Deming Cycle. 



Following is the application of the evaluation model we used to evaluate the 
Academic Performance Report. 



Frenchtling, et al. Evaluation Model 


Action Taken 


1 . Develop Evaluation Questions 


Figure out how best to evaluate the report. 


2. Match Questions with Appropriate 
Information-Gathering Techniques 


Created a survey to ask our readers about 
what information they used. 



O 

ERIC 



14 



16 



3. Collect Data 


Sent out a two page questionnaire to our 
report recipients with a follow-up phone 
call. 


4. Analyze Data 


Analyze survey results. 


5. Provide Information to Interested 
Audiences 


Summarized the survey results and shared 
with colleagues and report recipients. 



Using SAS to Streamline Report Preparation 

SAS is an invaluable tool for institutional researchers and can be used in multiple 
ways for reporting (Kunselman, 1997; Williams, 1996). SAS has played a significant 
role in streamlining our report preparation process. A relatively new feature of SAS and 
recently available to us is SAS/Connect. This product allows direct access to mainframe 
da ta while using SAS on a PC (Gamer & Polzin, 1996). Utilizing this capacity of SAS 
has allowed us to prepare a final report in one step rather than multiple steps. 

In the past, the method for preparing the retention report, which is a combination 
of table and graphs, was to run a SAS program against a mainframe file, transfer the 
mainframe output to a word document, and then manually enter data into Harvard 
Graphics. Now SAS/Connect is used to prepare the tables, by running a program against 
the mainframe file from the PC and directly using the SAS output as the report. This has 
simplified the procedure immensely by reducing the time for preparing the report and 
removing the possibility of errors from manual manipulation to the tables. 

The preparation of the graphic sections of the retention report was also 
streamlined. Originally, the data were exported from SAS PC to Excel and linked to 







Harvard Graphics. However, the version of Harvard Graphics available to us could not 
import the data into our version of Excel. The problem was solved with the new version 
of Harvard Graphics, which was able to link to Excel. (It is our goal to improve the 
proofing procedure by using Excel as a check of the data.) In Excel we created formulas 
for the calculations instead of hand calculation as before. The next step in our effort to 
continually improve our report preparation, will be to export the data from SAS into 
Excel. As shown, the streamlining of this report is a perpetual process. 

Continuous Improvement 

As we use the Deming Cycle and our own method to strive for never-ending 
improvement, we are also looking for new technology to utilize. SAS/Share, which is i 
currently being tested at our institution, will be implemented in the near future. . 
SAS/Share serves two basic pinposes. SAS/Share allows access of multiple users to the 
same SAS datasets simultaneously on the mainframe. Another benefit of SAS/Share is 
the ability to utilize other software, such as Brio, to access a SAS dataset on the 
mainfiame. This would allow Brio users to access current data to prepare reports more 
quickly and easily without having to develop the more demanding programming skills 
needed in SAS. 

Summary 

Receiving accurate and timely reports is critical for every institution of higher 
education. Creating a process that utilizes appropriate technology to reduce the time 
allocated for report preparation helps meet university goals of doing more with less. 
Following the Deming Cycle -- Plan, Do, Check, Act, being ever mindful of never- 
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ending quality improvement has been a useful model. Adapting this model for our 
specific needs in streamlining the reporting process has been a successful experience. 

The process used to streamline our student report preparation process has reduced the 
amount of labor needed for creating reports and improved the accuracy and timeliness of 
the reports. Our shared vision has deepened the commitment among staff members for 
our commonly held purpose. Surveying our users has given us information that has been 
used to better serve their needs and helped us understand how to improve the quality of 
our reports. 

As institutional researchers we understand the importance of always thinking of 
better approaches for preparing and presenting data to meet the needs of decision-makers. 
Streamlining reports is one way in which we can work smarter, not harder. 
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